Remote vehicles such as robots can be used in a variety of applications that would benefit from the ability to effectively collaborate with humans, including search-oriented applications (e.g., de-mining, cave exploration, foraging), rendering improvised explosive devices (IEDs) safe, and various other intelligence, surveillance and reconnaissance (ISR) missions. In addition, given then ability to effectively collaborate with humans, remote vehicles could be used in applications that require collaboration-oriented taskings in which is utilized member of a human/robot team, such as, for example, building clearing. Utilizing remote vehicles in building clearance and other similar tactical missions would help keep humans out of harm's way.
Remote vehicle and human teams performing tightly coordinated tactical maneuvers can achieve high efficiency by using the strengths of each member. Remote vehicle strengths include expendability, multi-modal sensing, and never tiring; while humans have better perception and reasoning capabilities. Taking advantage of these strength sets requires tight coordination between the humans and remote vehicles, with the remote vehicles reacting in real-time or near real-time to dynamically changing events as they unfold. The remote vehicle should also understand the goal and intentions of human team members' actions so that they can respond appropriately.
Having a human team member controlling the remote vehicles with a joystick during dynamic tactical maneuvers is less than ideal because it requires a great deal of the controlling human's attention. To enable a human operator to perform tactical maneuvers in conjunction with remote vehicles, the operator should be unencumbered and untethered and able to interact—to the greatest extent possible—with the remote vehicle as he/she would with another human teammate. This means the operator should have both hands free (e.g., no hand-held controllers) and be able to employ natural communication modalities such as gesture and speech to control the remote vehicle. Thus, it is desirable for remote vehicles to interact with their human counterparts using natural communication modalities, including speech and speech recognition, locating and identifying team members, and understand body language and gestures of human team members.